A video decoder receives encoded video data and decodes and/or decompresses the video data. The decoded video data comprises a series of pictures. A display device displays the pictures. The pictures comprise a two-dimensional grid of pixels. The display device displays the pixels of each picture in real time at a constant rate. In contrast, the rate of decoding can vary considerably for different video data. Accordingly, the video decoder writes the decoded pictures in a picture buffer.
Among other things, a display engine is synchronized with the display device and provides the appropriate pixels to the display device for display. The display engine determines the picture buffer address of the appropriate pixels, in order to provide the appropriate pixels to the display device. The address of the appropriate pixels is dependent on, among other things, the chroma format of the decoded pictures, the chroma format of the display device, the storage format, whether the pictures are interlaced or progressive, and whether the display is interlaced or progressive.
The chroma format of pictures describes the spatial relationship of chroma pixels to the luma pixels. A picture comprises a two dimensional grid of luma pixels Y, chroma red difference pixels Cr, and chroma blue difference pixels Cb. Generally speaking, the luma pixels are the greatest in number, and often as many, if not more, than the chroma Cr, and chroma Cb pixels combined. The grid of chroma Cr pixels and the grid of chroma Cb pixels are overlayed the grid of luma pixels during display. The overlaid grid of chroma pixels Cr/Cb can have a variety of spatial relationships with the luma pixels. For example, every other luma pixel can be co-located with a chroma Cr and chroma Cb pixel. Alternatively, the chroma pixels Cr/Cb may not align with the luma pixels Y. Examples of chroma formats include MPEG-1/2/4, 4:2:0 (now referred to as MPEG 4:2:0), MPEG 2/4 4:2:2 (now referred to as MPEG 4:2:2), DV-25, and MPEG-2 TM5.
Where the chroma format of the decoded pictures (decoded chroma format) and the chroma format of the display pictures (display chroma format) are different, chroma pixels corresponding to chroma pixel positions in the display chroma format are interpolated from the chroma pixels in the decoded picture that are nearest or surround the chroma pixel positions. During display, the chroma pixels in the decoded frame are replaced by the interpolated chroma pixels that are displayed in their corresponding chroma pixel positions in the display chroma format. One advantage of the planar format is that reading consecutive memory locations produces a rasterized bitstream of either the luma, chroma CrCr, or chroma Cb pixels.
The video decoder can also store a decoded picture in a number of different ways. In the planar format, the luma pixels are stored in raster order in one array, the chroma Cr pixels are stored in raster order in a second array, and the chroma Cb pixels are stored in raster order in a third array. In the packed YUV format, the luma, chroma CrCr, and chroma Cb pixels are packed together in a single array in raster order. One advantage of the packed YUV format is that reading consecutive memory locations produces a rasterized bitstream of the luma, chroma CrCr, and chroma Cb pixels.
Another format for storing decoded frames is known as the macroblock format. Several encoding and compression standards provide for a picture to be divided into two-dimensional blocks of luma pixels, chroma Cr pixels Cr, and chroma Cb pixels Cb. A macroblock comprises a block of luma Y pixels, and the blocks of chroma CrCr/Cb pixels are co-located therewith. The macroblocks are encoded, compressed, and represented by a data structure. The picture in its encoded and compressed form, is represented by the data structures representing the constituent macroblocks.
During decoding, the macroblocks are decoded/decompressed. As each macroblock is decoded, the video decoder stores the macroblock into a memory. A consecutive set of memory locations in one array store the block of luma pixels, and a consecutive set of memory locations in another array store the blocks of chroma Cr and chroma Cb pixels together.
Unlike planar storage format, or packed YUV storage format, however, reading consecutive memory locations where the macroblock format is used does not produce a rasterized bitstream.
The decoded pictures and the display can either be interlaced or progressive. Interlaced pictures are captured in two fields at two distinct time instances. All even numbered lines in the picture are captured/displayed at one discrete time, and all odd-numbered lines in the picture are captured/displayed at another discrete time. Both the National Television Standards Committee (NTSC) and Phase Alternate Lining (PAL) use interlacing. In contrast to interlaced pictures, progressive pictures are presented at the same discrete time instance.
Progressive display units are becoming more and more common. Most computer monitors are progressive display devices. Additionally, many television sets are capable of both interlaced and progressive displaying because more of the content displayed on television screens is from a progressive video sequence. For example, most motion pictures on Digital Versatile Discs (DVDs) are a progressive scan video sequence. Therefore, television sets can be equipped to display the DVD content as a progressive sequence. Additionally, many of the proposed high-definition television standards (HDTV) involve both progressive and interlaced displaying.
Conventionally, after each horizontal synchronization pulse, the host processor calculates the address of the first pixels of a line and the parameters for chroma format conversion after each horizontal synchronization pulse. The host processor then programs the display engine with the foregoing. Programming the display engine at each horizontal synchronization pulse consumes considerable bandwidth from the host processor.
An inherent problem exists when displaying an interlaced video on a progressive display. Most proposed solutions involve processing and analyzing the video signal in both the spatial and temporal domains, and producing a converted progressive picture based on the interlaced video source. Various methods for approaching the problem involve the utilization of vertical filters, vertical-temporal filters, adaptive two-dimensional and temporal filters, motion adaptive spatio-temporal filters, and motion-compensated spatio-temporal filters.
Very complex deinterlacers analyze the picture information in the spatial and temporal domains, sometimes storing several consecutive fields of video in memory in order to analyze the characteristics of the video sequence and make decisions on a pixel-by-pixel basis as to how to display the video information in a progressive display. Very simple deinterlacers perform only spatial filtering regardless of motion in the sequence. However, the foregoing look at actual picture data, and overlook the origin of the picture data, especially when the source is a compressed bit stream.
A particularly complex case arises when interlaced decoded pictures with a decoded chroma format are to be displayed on a progressive display with a different display chroma format.
The conventional approach in such a case is to convert the chroma format of the top field and bottom field, thereby generating a new top field and a new bottom field with the display chroma format. The new top field and bottom fields are provided to a deinterlacing filter. The deinterlacing filter operates on the new top field and the new bottom field to develop a progressive picture.
The foregoing approach has several disadvantages. The entire top field and bottom field are converted prior to deinterlacing. As a result, the advantages of pipelining are lost. Additionally, a large amount of data, e.g., the new top field and bottom field, are provided to the deinterlacer at one time. This causes increased reliance on software for data management, and the speed advantages of hardware are lost. Additionally, because the chroma format conversion does not occur in real-time, to guarantee completion of the chroma conversion and the interlaced to progressive conversion by display time, the chroma conversion occurs at a greater time prior to display. This reduces the processing power that is available to other functions that need to occur well in advance of display time.
Further limitations and disadvantages of conventional and traditional approaches will become apparent to one of skill in the art, through comparison of such systems with embodiments of the present invention as set forth in the remainder of the present application with reference to the drawings.